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[Claims] 

[Claim 1] High dynamic range image pickup device comprising a 
nondestructive image sensor having plural light -interception 
elements, read control means that repeats the reading process for a 
specific number of times while the image sensor is exposed, level 
comparison means that sequentially compares the read output level of 
each light-interception element of image sensor with the specific 
standard level, data memory means that stores a value corresponding 
to the read output level when the read output level of said light - 
interception element exceeds the standard level for the first time, 
counter memory means that stores a value corresponding to the number 
of reading repetition performed until the read output level exceeds 
the standard level for the first time, and computation means that 
acquires the image output by extrapolating the value corresponding to 
the output level stored in the data memory means based on the value 
corresponding to the specific reading count and repeated reading 
count stored in the count memory means . 

[Claim 2] High dynamic range image pickup device according to 
the claim 1, wherein said specific standard level is a value 
exceeding 1/2 of the saturation level of light-interception element. 

[Claim 3] High dynamic range image pickup device according to 
the claim 1 or 2 , wherein the number of read operations performed 
before the current reading operation is compared with the value 
corresponding to the repetition count stored in the count memory 
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means in order to determine the light-interception element output 
level having exceeded the standard level for the first time. 

[Detailed Explanation of this Invention] 

[0001] [Field of the Invention] 

The present invention relates to a high dynamic range image 
pickup device and is particularly associated with a device that can 
pick up an image (e.g., still image or semi still image having 
significant luminance gap) within a wide dynamic range with excellent 
accuracy. 

[0002] [Prior Art] 

Fig. 5 is a diagram showing the configuration of known image 
pickup device including an image sensor. With this type of device, 
output signals generated from each picture element (i.e., light- 
interception element) configuring an image sensor corresponding to 
the image light guided to the image sensor 1 are sequentially read 
out, amplified by an amplification circuit 2, and inputted to A/D 
converter 3. This input signal is converted to digital data by the 
A/D converter 3 and stored in a memory 4. A control circuit 5 
performs control operations. For example, it sequentially reads out 
a signal from each light- interception element of image sensor 1 and 
writes it to each address of memory 4. For example, the image data 
stored in the memory 4 as described above is read out by a computer 

(not shown in the figure) , processed, and displayed on a display 
device . 
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[0003] [Problems to Be Solved by this Invention] 

This type of known image -pickup device reads out image signals 
generated by simultaneously exposing every light-interception element 
practically for a single time. Therefore, the dynamic range of image 
pickup device cannot be widened. 

[0004] The detail of this problem will be explained below with 
the reference of Fig. 6 exhibiting the relation between time and 
output level of light interception element of usual image sensor. 
That is, Fig. 6 shows the relation between time and output levels of 
light interception elements A, B, C, D that respectively receive 
image lights having different strengths. As shown in the figure, 
when the image sensor is read out at time T, the output V A/i of 
picture element A and the output V B ,i of picture element B can be read 
out in the normal manner. However, the output V c ,i of picture element 
C and the output V D/ i of picture element D cannot be accurately read, 
as they are below the noise level V 0 . Also, when the output is read 
at time 2T, although the output V B/2 of picture element B can be 
accurately read, the output V A , 2 of picture element A, which has 
reached the saturation level V 2 , cannot be accurately obtained. In 
addition, the output levels of picture elements C and D, which are 
below the noise level, cannot be accurately obtained for the reason 
described above. 

[0005] On the other hand, when the output is read at time NT by 
setting a sufficient accumulation time, although output V C/N of 
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picture element C and output V D , N of picture element D can be read 
out, output level V A , N of output A and output level V b ,n of output B, 
which reach saturation during the process, cannot be accurately read 
out . 

[0006] Therefore, the read-out method of known image pickup 
device is restricted by the saturation level for the high intensity 
direction (i.e., direction of bright light), whereas the output of 
low intensity direction (i.e., direction of weak light) is limited by 
the noise level. Hence, the maximum dynamic range of image pickup 
device controlled by the dynamic range of the light interception 
element of image sensor is limited to approx. several thousands times 
at most. 

[0007] The object of present invention is to provide an image 
pickup device that can obtain a wide dynamic range without being 
restricted by the dynamic range of light interception element of 
image sensor, so as to solve the problems of the known devices. 
[0008] [Method to Solve the Problems] 

To achieve the object as described above, the present invention 
provides a high dynamic range image pickup device comprising a 
nondestructive image sensor having plural light -interception 
elements, read control means that repeats reading for a specific 
number of times while the image sensor is exposed, level comparison 
means that sequentially compares the read output level of each light - 
interception element of image sensor with a specific standard, level, 
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data memory means that stores the value corresponding to the read 
output level when the read output level of said light -interception 
element exceeds the standard level for the first time, counter memory 
means that stores the value corresponding to the number of repeated 
reading operations performed until the lead output level exceeds the 
standard level for the first time, and computation means that 
acquires the image output by extrapolating the value corresponding to 
the output level stored in the data memory means based on the value 
corresponding to a specific reading count and the number of repeated 
read operations stored in the count memory means. 
[0009] [Operation] 

With the configuration as described above, the readout control 
means reads out each light interception element for a specific number 
of times while the image sensor is exposed. Then, the level 
comparison means compares the readout output level of each light 
interception element with a standard level, and when the readout 
output level exceeds the standard level for the first time, a value 
corresponding to the readout output level is stored in the data 
memory means, and the value corresponding to the number of previous 
readouts is stored in the repetition count memory means. Then, with 
the completion of readouts performed for a specific number of times, 
the computation means acquires the image output by extrapolating the 
value corresponding to the output level stored in the data memory 
means based on the value corresponding to a specific read-count and 
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the number of repeated read operations stored in the count memory 
means . 

[0010] With this technique, an output exceeding the noise level 
can be obtained by setting sufficient accumulation time for extremely 
low intensity signals. Also, for a picture element receiving high 
intensity light to cause saturation during the accumulation process, 
a value reflecting the true output level can be obtained by 
extrapolating the data prior to the saturation stored in the data 
memory. Therefore, the dynamic range can be drastically widened 
regardless of limitation of light interception element 
characteristic . 
[0011] [Operational Example] 

The following explains the operational example of this invention 
while referring to the figures. Fig. 1 is a diagram showing the 
configuration of high dynamic range image pickup device used in the 
Operational example 1 of this invention. The device shown in the 
figure comprises an image sensor 11, amplification circuit 12, A/D 
converter 13, data memory 14, count memory 15, counter 16, comparison 
circuit 17, multiplication/division circuit 18, and control circuit. 

[0012] The. image sensor 11 is a nondestructive sensor and 
contains numerous light interception elements. The amplification 
circuit 12 is for buffering and amplifying the output of the image 
sensor 11. The A/D converter 13 sequentially converts the readout 
outputs of the image sensor being amplified by the amplification 
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circuit 12 to digital values. The data memory 14 is a memoary device 
that stores values for each picture element based on the digitized 
readout output level provided by the A/D converter based on the 
output from the comparison circuit 17. The counter memory 15 is 
controlled according to the output of the comparison circuit 17 and 
stores the number of readouts for each picture element, counted until 
the output level of light interception element exceeds a specific 
value. The counter 16 counts the readouts. The comparison circuit 

17 compares the digitized readout output level of light interception 
element with a specific value. The multiplication/division circuit 

18 is a computation circuit that multiplies and divides for computing 
the final output. The control circuit 19 supplies a clock pulse and 
control signal to the circuit block and controls their operations. 

[0013] The following will explain the image pickup operation as 
described above by referring to the flowchart shown in Fig. 2. Prior 
to reading out of image sensor, the control circuit 19 sends a reset 
signal so as to set the counter 16 to zero and stores the maximum 
repetition count N to the area corresponding to every light 
interception element. 

[0014] Next, the control circuit 19 sends an increment signal 
and increment the counter 16 by 1 . That is, the counter 16 is set to 
1, indicating the first readout. In this condition, the output of 
each picture element of image sensor 11 is sequentially read out, and 
the readout signal is amplified by the amplification circuit 12. 
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Then, the data is converted to a digital value using the A/D 
converter . 

[0015] The output digital value of A/D converter 13 is expressed 
as Vj fk , which is compared with the standard value V x by the comparison 
circuit 17. Note that 'j' denotes the picture element number of 
image sensor 11, and *k' denotes the readout count. That is, Vj # k is 
the *k'th output of the 'j'th picture element. For example, when the 
comparison result of comparison circuit 17 indicates that the output 
Vj, k exceeds the standard value V x as was the case of first readout 
output V A/ i shown in Fig. 6, the readout count k (1 in this example) is 
stored in the area corresponding to the readout light interception 
element of the count memory. That is, the value is set as Nj = k. In 
this case, Nj is the data of "j'th picture element stored in the count 
memory. Furthermore, this readout output value V A/i is stored in the 
area corresponding to the picture element A of the data memory 14 
(i.e., Mj = Vj /k ) , whereas Mj is the data in the data memory 14 of the 
jth picture element. 

[0016] On the other hand, when the first output does not exceed 
the standard value V lf such as picture elements B, C, and D in Fig. 6/ 
the respective output values V B/1 , V C/i/ V D/1 , of picture elements B, C, 
and D are stored to the corresponding areas in the data memory 14 . 
In this case, since writing is not performed to the corresponding 
area in the count memory 15, the maximum readout count N is left in 
the memory. With the completion of readout for every picture 
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element, 1 is added to the counter 16 to change the counter value to 
2 for starting the second readout. 

[0017] During the second readout, the output of light 
interception element of each picture element of image sensor 11 and 
count memory 15 are sequentially read. For example, although the 
output V A/2 of the picture element A shown in Fig. 6 exceeds the 
standard value Vi, the value N A in the corresponding area in the 
counter memory 1 has been set to "1" by the initial readout, and does 
not contain N. Therefore, data is not stored in the data memory 14 
or counter memory 15 . 

[0018] On the other hand, when the readout output V B/2 of picture 
element B exceeds the standard value Vi, since the area in the counter 
memory 15 corresponding to the picture element B has N, the readout 
count k=2 is stored in the corresponding area in the counter memory 
15. That is, the stored data V B in the area corresponding to the 
picture element B of counter memory 15 becomes M 2". 

[0019] However, since the readout output values V c ,2 and V D(2 of 
picture elements C, D in Fig. 6 do not exceed the standard value V lf 
those output values V Cf2 and V D/2 are stored to the corresponding area 
of the data memory 14 . Note that the stored values in the counter 
memory 15 are not modified. 

[0020] The operation described above is repeated for N times by 
sequentially incrementing the value of the counter 16. As a result, 
Va,i V B/2 V c ,n/ V d , n are stored in the corresponding areas of data memory 
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14 for each picture element A, B, C, and D at the end of the process, 
whereas 1, 2, N, and N are stored in the counter memory 15. 

[0021] Next, the multiplication/division circuit 18 computes the 
output of each picture element. That is, when the output of picture 
element j is Vj, the following formula can be used to compute the 
output : 

Vj = (N/N-j) -Mj 

As shown in Fig. 3, this formula obtains the output Vj at Nth 
readout (after the exposure time NT) by extrapolating the output Mj 
stored in the data memory. 

[0022] In the example shown in Fig. 6, the final output V A of 
picture element A becomes V A ,i'N, and the final output V B of picture 
element B becomes V B/2 'N/2. Also, the final output V c of picture 
element C becomes V C ,2'N, and the final output V s of picture element D 
becomes V D/N . 

[0023] Since the output of picture element having saturated 
during accumulation can be obtained by extrapolation, the dynamic 
range of image sensor itself can be drastically (i.e., to N times) 
expanded. For example, if N=1000, the dynamic range which is several 
thousands times greater than the range of image sensor itself can be 
expanded for thousand times, thus acquiring the dynamic range 
expanded for several million times. 

[0023] Also, the standard value V! is preferably set to approx. 
1/2 of the saturated output level V 2 . This is because the level 
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allowing the prediction of exceeding the saturation point at the next 
read out when the output level exceeds the standard value exceeds Vi 
is used as a standard level. Without saying that the standard level 
is not limited to approx. 1/2 of the saturation level V 2 , as any level 
is applicable. 

[0024] Fig. 4 is a diagram showing the configuration of high 
dynamic range image pickup device used in the operation al example of 
this invention. With the device shown in the figure, the output of 
image sensor 11 is inputted to a data processing device such as 
microcomputer 20 via an amplification circuit 12 and A/D converter. 
The microcomputer 20 is operated by the internal program executing 
the procedures shown in Fig. 2. 

[0025] The device shown in Fig. 4 is conveniently used for the 
application such as astronomic observation that requires a lengthy 
total accumulation period for the image sensor, and the readout speed 
of the first exposure may be slow. This application can simplify the 
circuit configuration compared with the device shown in Fig. 1. 
[0026] [Effectiveness of this Invention] 

With the method based on this invention explained above, when 
the signal is weak, an output higher than the noise level can be 
obtained by setting a longer accumulation period. Also, for strong 
signals, the final output can be obtained by extrapolating data based 
on the value obtained within the short accumulation time prior to the 
saturation of interception element output. Since the essential 
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dynamic range of image sensor can be drastically expanded by this 
technique, an object with a significant luminous intensity gap can be 
accurately picked up. 
[Simple Explanation of the Figures] 

[Figure 1] is a diagram showing the configuration of high 
dynamic range image pickup device used in the Operational example 1 
of this invention. 

[Figure 2] is a flowchart explaining the operation of image 
pickup device shown in Fig. 1. 

[Figure 3] is a diagram exhibiting the computation method of 
final output performed by the multiplication/division circuit of the 
device shown in Fig. 1. 

[Figure 4] is a diagram showing the configuration of high 
dynamic range image pickup device used in the Operational example of 
this invention. 

[Figure 5] is a diagram showing the known configuration of image 
pickup device including an image sensor. 

[Figure 6] is a graph exhibiting the relation between time and 
output levels of light interception elements. 
[Explanation of Keys] 

11... Image sensor; 12. ..Amplification circuit; 13...A/D converter; 
14. ..Data memory; 15. ..Counter memory; 16. ..Counter; 17. ..Comparison circuit; 
18. ..Multiplication/division circuit; 19. ..Control circuit; 20. ..Micro 
computer. 
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Figure 1 
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Key A.. .Start; EL.End; l...Read out image sensor and set A/D conversion 
output to Vj,k; 2. ..Store readout count k to count memory; 3. ..Store Vj , k 
to data memory; 4. ..Was every picture element read?; 5. ..Repeated for N 
times?; 6. ..Output computation 



15 



Figure 4 
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